Skip to content

Admin showcase panel for signed-in users - #1137

Merged
northdpole merged 119 commits into
stagingfrom
admin-panel-showcase
Nov 1, 2026
Merged

northdpole merged 119 commits into
stagingfrom
admin-panel-showcase

Conversation

@northdpole

Copy link
Copy Markdown
Collaborator

Admin IA redesign stacked on the OIE / OWASP-agent tip (cursor/oie-scheduled-runs-eb24, PR #1132). When we merge, merge the whole stack — not this PR onto main alone.

Mac handover

See docs/oie/admin-e2e-mac-handover.md — open questions first, then local run / API / E2E sequence. Continue validation on Mac, not cloud agent.

git fetch origin cursor/admin-panel-showcase-ccdd && git checkout cursor/admin-panel-showcase-ccdd && git pull
SKIP_UPSTREAM_SYNC=1 PORT=5001 make dev

What changed

  • make install / make dev → Docker Postgres + upstream CRE when empty
  • Admin Config: golden-set harvest + OWASP agent sync buttons
  • Graph JSON errors; OIE skip/flag observability
  • Default admin ingest still skips B/C unless skip_b/skip_c false (full E2E must override + LLM keys)

Explorer after upstream sync
explorer-after-upstream-sync.mp4

To show artifacts inline, enable in settings.

Open in Web Open in Cursor 

Made with Cursor

@coderabbitai

coderabbitai Bot commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

🗂️ Base branches to auto review: 1
  • main

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@northdpole
northdpole force-pushed the cursor/oie-scheduled-runs-eb24 branch from 5236f80 to 061f632 Compare October 11, 2026 23:04
@northdpole
northdpole force-pushed the admin-panel-showcase branch from e114292 to 4178ecf Compare October 11, 2026 23:04
@northdpole
northdpole force-pushed the cursor/oie-scheduled-runs-eb24 branch from 061f632 to 719716d Compare October 12, 2026 07:05
@northdpole
northdpole force-pushed the admin-panel-showcase branch from 4178ecf to aad929a Compare October 12, 2026 07:05
@northdpole
northdpole force-pushed the cursor/oie-scheduled-runs-eb24 branch from 719716d to ae89728 Compare October 12, 2026 07:15
@northdpole
northdpole force-pushed the admin-panel-showcase branch from aad929a to 475945f Compare October 12, 2026 07:17
@northdpole
northdpole force-pushed the cursor/oie-scheduled-runs-eb24 branch from ae89728 to 6e444d3 Compare October 12, 2026 07:18
@northdpole
northdpole force-pushed the admin-panel-showcase branch 2 times, most recently from 48e9d4f to 30a2a40 Compare October 12, 2026 22:19
Dual-index retrieval, CRE summaries, prior cage, hub seed, edition remap,
and related librarian defaults. Rewrites #1088 as the reviewable base; later
stacked PRs add harvester, B2 eval, fixtures, mappings, and full-pipeline docs.
- Refuse plain-text fetch redirects that leave GitHub raw hosts (SSRF).
- Re-check expected embedding dimension on query-cache hits.
- Fill phrase-only oie blocks in apply_oie_fill_if_missing without losing phrases.
- Expose last_preferred_cre_ids on StandardHopRetriever and honour its allowlist.
- Commit each metadata enrichment so a later failure cannot roll back earlier work.
- Load .env from the repo root in the enrichment scripts.
- Parse hostnames for CRE hub sync SSL and loopback checks; batch SQLite IN
  lookups; stop echoing database credentials.
Optional requirement extract (off/auto/on) plus GitHub tarball/URL ingest
wiring for Module B → C. Stacked on #1088 librarian base.
Harness runners, hop-distance scoring, and gold loaders without bulky
source fixtures (those land in follow-up stacked PRs).
Wire LlmSafetyGuard (LiteLLM) as the live factory default so runs evaluate
safety instead of NullSafetyGuard. Apply the guard on every decision path,
prefer safety reason codes over empty shortlists, and cover a synthetic
adversarial corpus in unit tests.
- Do not fall back to main/master when the user named a branch.
- Download tarballs with a timeout and a size cap; cap per-member and total
  markdown bytes read from archives.
- Validate owner, repo and branch and URL-encode them for codeload.
- Delete prior ingest rows only after download and chunking succeed, and never
  rows from other runs.
- Pass the model override through a local config instead of os.environ.
- Redact database passwords in orchestrator, B2 and setup_oie output; write
  tmp/oie.env with mode 600.
- Keep the existing --db for --skip-a and a user-supplied --repos-yaml.
- Create the artifact dir before writing orchestrator reports.
- Clear per-model knowledge rows between Qwen and Gemini classify runs.
- Key the switch-grid embed cache by model, honour --force with stale-lock
  recovery, write explicit zero flags to stack_winners.yaml, and tolerate
  LibrarianConfig fields that are env-only.
- Match B1 sheet-grain gold exactly instead of by substring.
Large fixture slice for ASVS eval arms only — split from NIST/other sources
so reviewers can treat this as mechanical data review.
NIST OSCAL sample plus Top10/API/K8s/LLM/AISVS/agentic fixture sources and
gold JSON. Stacked after the ASVS-only fixture PR.
Update OWASP mapping fixtures and loader helpers so orphan umbrella Related
targets become canonical gold for eval and product mappings.
- Forward leaf_drilldown_min_sections=0 instead of replacing it with 20.
- Make queue wiping in run_full_pipeline opt-in via --wipe-queues and add the
  flag to the detached clone-DB runner, which now reports its real exit code.
- Fall back to sys.executable when venv/bin/python is missing.
- Fix the B2 report gate string (top-3) and drop the nonexistent make target
  from the docs.
- Make the denied-resource drill-down test actually reach the allowlist gate.
Prior-caged retrieval embeds the same header and body several times per chunk. Prefetch the unique strings for a queue batch so those Gemini calls collapse into one list request.
Makefile/full-pipeline glue, OIE human docs, and experiment YAMLs. Experimental
flags are documented as removable on merge if they are not useful to keep.
- Drop scripts/oie_owasp_eval/fixtures/reports/*.json: generated accuracy
  output that no code reads (module_c_report.json also embedded an absolute
  local path). Numbers are kept in the project's session records.
- Drop the five b2_gold overlays (owasp_aisvs_1_0, owasp_api_top10_2023,
  owasp_kubernetes_top10_2022, owasp_kubernetes_top10_2025, owasp_top10_2025).
  They are not byte-identical to the canonical mapping fixtures but are
  regenerable hub-expanded overlays (rebuild_b2_gold_from_hub.py), and
  load_gold() reads only the agentic stub from b2_gold. All seven harnesses
  still load. owasp_agentic_ai_stub.json and owasp_llm_top10_2025.json stay.
* trim/1118:
  chore(oie): trim #1118 for the staging rollout
  chore(oie): trim #1117 for the staging rollout
  chore(oie): trim #1116 and guard destructive eval scripts
  chore(harvester): trim #1115 for the staging rollout
  chore(oie): trim #1088 for the staging rollout
* trim/1119:
  chore(oie): trim #1118 for the staging rollout
  chore(oie): trim #1117 for the staging rollout
  chore(oie): trim #1116 and guard destructive eval scripts
  chore(harvester): trim #1115 for the staging rollout
  chore(oie): trim #1088 for the staging rollout

# Conflicts:
#	requirements-dev.txt
Only file in the repo that failed 'black --check'. Pure line wrap.
…ad experiments

- Resolve leftover conflict markers in librarian factory/pipeline/queue_runner
  (SyntaxError on import); verified against the staging working versions.
- run_full_pipeline.py refuses non-local DBs and the database named 'cre'
  (it empties harvest_input/knowledge_queue/decision_queue); default
  cre_prodclone; remove dead helpers.
- Drop run_d1_lever_experiments.py, run_d1_levers_detached.sh,
  run_fullpipe_detached.sh, tarball_harvest.py and the rrf/context_enrich
  experiment configs (removed from the stack; preserved on
  staging-experiments, see experiments ledger).
- Docs: removed-flag rows, correct CLI flags (--cache_file/--out-dir),
  local-clone requirement, drop non-existent make target.
* trim/1120:
  trim(#1120): fix merge-conflict markers, guard full pipeline, drop dead experiments
  style: black-format llm_safety_guard.py
  chore(oie): trim #1118 for the staging rollout
  chore(oie): trim #1117 for the staging rollout
  chore(oie): trim #1116 and guard destructive eval scripts
  chore(harvester): trim #1115 for the staging rollout
  chore(oie): trim #1088 for the staging rollout

# Conflicts:
#	application/utils/librarian/factory.py
#	docs/oie/full-pipeline.md
#	scripts/oie_owasp_eval/run_fullpipe_detached.sh
* trim/1125:
  trim(#1120): fix merge-conflict markers, guard full pipeline, drop dead experiments
  style: black-format llm_safety_guard.py
  chore(oie): trim #1118 for the staging rollout
  chore(oie): trim #1117 for the staging rollout
  chore(oie): trim #1116 and guard destructive eval scripts
  chore(harvester): trim #1115 for the staging rollout
  chore(oie): trim #1088 for the staging rollout
…ow-ups, set source_channel in raw embedding writers

- Remove scripts/owasp_agent_adversarial_asker.py and ..._batch_run.py
  (hardcoded homeserver ssh target, import a module and MCP tool that
  do not exist).
- Remove _maybe_append_cre_followups: it changed every CRE chat answer
  regardless of OWASP_AGENT_ENABLED.
- oie_sync_cre_hub.py and sync_embeddings_table.py insert
  embeddings.source_channel (cre_graph / corpus); without it the cre_graph
  lane in lane_similarity.py is empty after a hub sync.
- isolation_mcp_test: the opt-in meta-embedding push legitimately uses
  Node_collection; allow exactly that one use behind OWASP_AGENT_EMBED_META.
- black-format scripts/owasp_agent_synth_eval.py and owasp_agent_embed_meta.py.
* trim/1127:
  trim(#1127): drop adversarial asker/batch scripts and canned CRE follow-ups, set source_channel in raw embedding writers
  trim(#1120): fix merge-conflict markers, guard full pipeline, drop dead experiments
  style: black-format llm_safety_guard.py
  chore(oie): trim #1118 for the staging rollout
  chore(oie): trim #1117 for the staging rollout
  chore(oie): trim #1116 and guard destructive eval scripts
  chore(harvester): trim #1115 for the staging rollout
  chore(oie): trim #1088 for the staging rollout

# Conflicts:
#	cre.py
#	scripts/owasp_agent_adversarial_asker.py
#	scripts/owasp_agent_adversarial_batch_run.py
…README/dev setup, loopback guard for run_admin_local.sh

- Remove .cursor/skills/brevity-bot, review-brevity, brevity_bot_skill_test.py
  and the matching .gitignore re-includes (unrelated to the admin panel).
- Remove docs/oie/admin-e2e-mac-handover.md (wrong base/PR, contradicts
  _default_skip_bc()).
- Restore README.md and docs/developmentSetup.md (the rewrite removed the
  SQLite contributor path).
- scripts/run_admin_local.sh refuses non-loopback Postgres hosts
  (ADMIN_LOCAL_ALLOW_REMOTE=1 to override) since it runs flask db upgrade,
  create_all and NO_LOGIN=1.
- sync.py: undo the getattr('Node'+'_collection') trick that evaded the
  isolation guardrail; the guardrail test now states the one allowed use.
…nto embed_sync

sync_all no longer imports Node_collection/PromptHandler, so
test_sync_never_imports_cre_defs holds again without an allowance.
embed_meta_if_enabled(store, report, *, database=None, embed_fn=None) applies the
same OWASP_AGENT_EMBED_META flag and logging; callers invoke it after sync_all.
upsert_channel_embedding also refuses a vector whose length differs from the
dimension of the existing embeddings rows when CRE_EMBED_EXPECTED_DIM is unset.
README no longer claims a separate SQLite index or that embeddings are never
written.
* trim/1127:
  chore(owasp-agent): type embed_sync and its test
  chore: drop generated frontend bundle (rebuild at deploy)
  refactor(owasp-agent): move the meta-embedding push out of sync_all into embed_sync

# Conflicts:
#	application/frontend/www/bundle.js
…a-embedding push

sync_all no longer embeds; call embed_meta_if_enabled after it so deployments
that set OWASP_AGENT_EMBED_META=1 keep embedding.
* trim/1127:
  test(owasp-agent): sync and concepts must not import application.database
* trim/1127:
  test(owasp-agent): restore the environment with a plain helper instead of patch.dict in the embed_sync tests
northdpole added a commit that referenced this pull request Nov 1, 2026
* trim/1137:
  trim(#1137): drop brevity-bot skills and stale handover doc, restore README/dev setup, loopback guard for run_admin_local.sh
  trim(#1127): drop adversarial asker/batch scripts and canned CRE follow-ups, set source_channel in raw embedding writers
  trim(#1120): fix merge-conflict markers, guard full pipeline, drop dead experiments
  style: black-format llm_safety_guard.py
  chore(oie): trim #1118 for the staging rollout
  chore(oie): trim #1117 for the staging rollout
  chore(oie): trim #1116 and guard destructive eval scripts
  chore(harvester): trim #1115 for the staging rollout
  chore(oie): trim #1088 for the staging rollout
northdpole added a commit that referenced this pull request Nov 1, 2026
* trim/1136:
  trim(#1137): drop brevity-bot skills and stale handover doc, restore README/dev setup, loopback guard for run_admin_local.sh
  trim(#1127): drop adversarial asker/batch scripts and canned CRE follow-ups, set source_channel in raw embedding writers
  trim(#1120): fix merge-conflict markers, guard full pipeline, drop dead experiments
  style: black-format llm_safety_guard.py
  chore(oie): trim #1118 for the staging rollout
  chore(oie): trim #1117 for the staging rollout
  chore(oie): trim #1116 and guard destructive eval scripts
  chore(harvester): trim #1115 for the staging rollout
  chore(oie): trim #1088 for the staging rollout
  test: embedding-failure scheduler test needs graph filing enabled now that it defaults off
  style: format files flagged by CI (black, prettier)
  Make event-repo harvesting opt-in (OIE_HARVEST_EVENTS)
  Make event page parsing linear-time and clean Playwright fallback text
  Embed auto-filed graph nodes in the CRE expansion job
  Parse OWASP event pages and harvest event repos

# Conflicts:
#	application/utils/oie_scheduler/jobs.py
#	scripts/oie_owasp_eval/run_full_pipeline.py
northdpole added a commit that referenced this pull request Nov 1, 2026
* trim/1142:
  trim(#1142): admin upstream-sync-and-ingest no longer replaces existing CRE embeddings
  trim(#1142): no credentialed CORS without an allowlist, OIE workers opt-in in import-all, guard levers-matrix queue reset
  trim(#1137): drop brevity-bot skills and stale handover doc, restore README/dev setup, loopback guard for run_admin_local.sh
  trim(#1127): drop adversarial asker/batch scripts and canned CRE follow-ups, set source_channel in raw embedding writers
  trim(#1120): fix merge-conflict markers, guard full pipeline, drop dead experiments
  style: black-format llm_safety_guard.py
  chore(oie): trim #1118 for the staging rollout
  chore(oie): trim #1117 for the staging rollout
  chore(oie): trim #1116 and guard destructive eval scripts
  chore(harvester): trim #1115 for the staging rollout
  chore(oie): trim #1088 for the staging rollout
  test: embedding-failure scheduler test needs graph filing enabled now that it defaults off
  style: format files flagged by CI (black, prettier)
  Make event-repo harvesting opt-in (OIE_HARVEST_EVENTS)
  Make event page parsing linear-time and clean Playwright fallback text
  Embed auto-filed graph nodes in the CRE expansion job
  Parse OWASP event pages and harvest event repos

# Conflicts:
#	scripts/owasp_agent_adversarial_asker.py
#	scripts/owasp_agent_adversarial_batch_run.py
northdpole added a commit that referenced this pull request Nov 1, 2026
* trim/1155:
  trim(#1155): drop laptop-local edge probe scripts, make node metadata seeding opt-in, fix empty-key blurb match
  trim(#1142): admin upstream-sync-and-ingest no longer replaces existing CRE embeddings
  trim(#1142): no credentialed CORS without an allowlist, OIE workers opt-in in import-all, guard levers-matrix queue reset
  trim(#1137): drop brevity-bot skills and stale handover doc, restore README/dev setup, loopback guard for run_admin_local.sh
  trim(#1127): drop adversarial asker/batch scripts and canned CRE follow-ups, set source_channel in raw embedding writers
  trim(#1120): fix merge-conflict markers, guard full pipeline, drop dead experiments
  style: black-format llm_safety_guard.py
  chore(oie): trim #1118 for the staging rollout
  chore(oie): trim #1117 for the staging rollout
  chore(oie): trim #1116 and guard destructive eval scripts
  chore(harvester): trim #1115 for the staging rollout
  chore(oie): trim #1088 for the staging rollout
  test: embedding-failure scheduler test needs graph filing enabled now that it defaults off
  style: format files flagged by CI (black, prettier)
  Make event-repo harvesting opt-in (OIE_HARVEST_EVENTS)
  Make event page parsing linear-time and clean Playwright fallback text
  Embed auto-filed graph nodes in the CRE expansion job
  Parse OWASP event pages and harvest event repos
northdpole added a commit that referenced this pull request Nov 1, 2026
* trim/1159:
  trim(#1159): drop cambridge->boston alias, dead locals, test env/tempfile leaks
  trim(#1155): drop laptop-local edge probe scripts, make node metadata seeding opt-in, fix empty-key blurb match
  trim(#1142): admin upstream-sync-and-ingest no longer replaces existing CRE embeddings
  trim(#1142): no credentialed CORS without an allowlist, OIE workers opt-in in import-all, guard levers-matrix queue reset
  trim(#1137): drop brevity-bot skills and stale handover doc, restore README/dev setup, loopback guard for run_admin_local.sh
  trim(#1127): drop adversarial asker/batch scripts and canned CRE follow-ups, set source_channel in raw embedding writers
  trim(#1120): fix merge-conflict markers, guard full pipeline, drop dead experiments
  style: black-format llm_safety_guard.py
  chore(oie): trim #1118 for the staging rollout
  chore(oie): trim #1117 for the staging rollout
  chore(oie): trim #1116 and guard destructive eval scripts
  chore(harvester): trim #1115 for the staging rollout
  chore(oie): trim #1088 for the staging rollout
  test: embedding-failure scheduler test needs graph filing enabled now that it defaults off
  style: format files flagged by CI (black, prettier)
  Make event-repo harvesting opt-in (OIE_HARVEST_EVENTS)
  Make event page parsing linear-time and clean Playwright fallback text
  Embed auto-filed graph nodes in the CRE expansion job
  Parse OWASP event pages and harvest event repos
northdpole added a commit that referenced this pull request Nov 1, 2026
* trim/1158:
  trim(#1158): strip_injection no longer truncates legitimate questions; correct default-off claims
  trim(#1159): drop cambridge->boston alias, dead locals, test env/tempfile leaks
  trim(#1155): drop laptop-local edge probe scripts, make node metadata seeding opt-in, fix empty-key blurb match
  trim(#1142): admin upstream-sync-and-ingest no longer replaces existing CRE embeddings
  trim(#1142): no credentialed CORS without an allowlist, OIE workers opt-in in import-all, guard levers-matrix queue reset
  trim(#1137): drop brevity-bot skills and stale handover doc, restore README/dev setup, loopback guard for run_admin_local.sh
  trim(#1127): drop adversarial asker/batch scripts and canned CRE follow-ups, set source_channel in raw embedding writers
  trim(#1120): fix merge-conflict markers, guard full pipeline, drop dead experiments
  style: black-format llm_safety_guard.py
  chore(oie): trim #1118 for the staging rollout
  chore(oie): trim #1117 for the staging rollout
  chore(oie): trim #1116 and guard destructive eval scripts
  chore(harvester): trim #1115 for the staging rollout
  chore(oie): trim #1088 for the staging rollout
  test: embedding-failure scheduler test needs graph filing enabled now that it defaults off
  style: format files flagged by CI (black, prettier)
  Make event-repo harvesting opt-in (OIE_HARVEST_EVENTS)
  Make event page parsing linear-time and clean Playwright fallback text
  Embed auto-filed graph nodes in the CRE expansion job
  Parse OWASP event pages and harvest event repos

# Conflicts:
#	application/mcp/ask.py
#	application/web/web_main.py
northdpole added a commit that referenced this pull request Nov 1, 2026
* trim/1162: (27 commits)
  docs(env): document CHAT_BLOCK_UNVERIFIED (default on, changes public chat answers)
  trim(#1162): drop prune_noise_chunks (unguarded DELETE on embeddings), validate authority weights before use, document ANSWER_LOG_*, no host paths in sync_check_staging
  trim(#1158): strip_injection no longer truncates legitimate questions; correct default-off claims
  trim(#1159): drop cambridge->boston alias, dead locals, test env/tempfile leaks
  trim(#1155): drop laptop-local edge probe scripts, make node metadata seeding opt-in, fix empty-key blurb match
  trim(#1142): admin upstream-sync-and-ingest no longer replaces existing CRE embeddings
  trim(#1142): no credentialed CORS without an allowlist, OIE workers opt-in in import-all, guard levers-matrix queue reset
  trim(#1137): drop brevity-bot skills and stale handover doc, restore README/dev setup, loopback guard for run_admin_local.sh
  trim(#1127): drop adversarial asker/batch scripts and canned CRE follow-ups, set source_channel in raw embedding writers
  trim(#1120): fix merge-conflict markers, guard full pipeline, drop dead experiments
  style: black-format llm_safety_guard.py
  chore(oie): trim #1118 for the staging rollout
  chore(oie): trim #1117 for the staging rollout
  chore(oie): trim #1116 and guard destructive eval scripts
  chore(harvester): trim #1115 for the staging rollout
  chore(oie): trim #1088 for the staging rollout
  fix(auth): harden web sign-in found by adversarial review
  fix(auth): /auth/user needs any role, not admin
  fix(auth): carry the PKCE code_verifier from login to callback
  fix(auth): build a fresh OAuth flow per login instead of a shared singleton
  ...
@northdpole
northdpole changed the base branch from fix/owasp-agent-probe-gaps to staging November 1, 2026 22:51
@northdpole
northdpole merged commit d67c1ff into staging Nov 1, 2026
11 checks passed
@northdpole
northdpole deleted the admin-panel-showcase branch November 1, 2026 22:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants